Should I disallow(robots.txt) archive/author pages with links already available on the front page? [on hold]

Posted by WPRookie82 on Pro Webmasters See other posts from Pro Webmasters or by WPRookie82
Published on 2014-08-21T19:19:43Z Indexed on 2014/08/21 22:28 UTC
Read the original article Hit count: 229

Filed under:

robots.txt

|

nofollow

I am working on a simple Wordpress blog where when an article is published, it appears on ALL these pages:

Homepage - Headline(clickable) + 3-line summary
Parent category page - Headline(clickable) + 3-line summary
Child category page - Headline(clickable) + 3-line summary
Author page - Headline(clickable)
sitemap.xml

I've been told that I should add all author pages to my robots.txt, under disallow, so as search engine bots do not spider /author/* since all links on these pages are available elsewhere.

Is this a good approach or maybe rel=nofollow is better, or maybe I shouldn't worry about this at all?

© Pro Webmasters or respective owner

Related posts about robots.txt

Robots.txt practices with .htaccess redirections (inherits)

as seen on Pro Webmasters - Search for 'Pro Webmasters'
I have a question regarding how to write robots.txt files for many domains and subdomains with redirects in place. We have a hosting account that enacts primary and add-on domains. All of our domains and subdomains, including the primary domain, is redirected via htaccess 301s to their own subdirectories… >>> More
mod evasive not working properly on ubuntu 10.04

as seen on Server Fault - Search for 'Server Fault'
I have an ubuntu 10.04 server where I installed mod_evasive using apt-get install libapache2-mod-evasive I already tried several configurations, the result stays the same. The blocking does work, but randomly. I tried with low limis and long blocking periods as well as short limits. The behaviour… >>> More
Cross-domain jQuery using YQL gives robots.txt error

as seen on Stack Overflow - Search for 'Stack Overflow'
On the page http://qxlapps.dk/test.htm I am trying to perform an Ajax load from another domain, qxlapp.dk. I am using James Padolsey's xdomainajax.js plugin from: http://james.padolsey.com/javascript/cross-domain-requests-with-jquery/ When I open my test page, I get no output, but FireBug shows… >>> More
Asterisk in robots.txt

as seen on Stack Overflow - Search for 'Stack Overflow'
Wondering if following will work for google in robots.txt Disallow: /*.action I need to exclude all urls ending with .action. Is this correct? >>> More
SEO chaos from changing robots.txt file in Wordpress site

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi there, I recently edited the robots.txt file in my site using a wordpress plugin. However, since i did this, google seems to have removed my site from their search page. I'd appreciate if I could get an expert opinion on why this is so, and a possible solution. I'd initially done it to increase… >>> More

Related posts about nofollow

What does rel="nofollow me" do?

as seen on Stack Overflow - Search for 'Stack Overflow'
I notice on SO userpage all urls have rel="nofollow me" (i was thinking of putting nofollow on my site so i checked here to see what SO does). What does the 'me' part do? >>> More
Not using nofollow for UGC links

as seen on Pro Webmasters - Search for 'Pro Webmasters'
My site contains many links that were user generated so I've added a rel=nofollow to them. I'm considering removing that tag at least partially, as these are most of my outgoing links and I've come to understand it isn't a good SEO sign to not have outgoing followed links. Although these are external… >>> More
NoFollow and DoFollow Blog - Comment Links Affect Search Engine Optimization

as seen on Ezine Articles - Search for 'Ezine Articles'
Many sellers of information products leave comments on blogs, especially popular ones with high Google PageRank, thinking that they are getting those good inbound links to their sites. But there's a problem. Most blogs put a "No Follow" tag in the link to your website. Sure, readers can click on it… >>> More
Dofollow and Nofollow in Link Building

as seen on Ezine Articles - Search for 'Ezine Articles'
Dofollow refers to links that do not have nofollow attribute. It is recognized as a vote and the link will be crawled by the search engine spiders. >>> More
NoFollow Tag Implementation and Its Impact

as seen on Ezine Articles - Search for 'Ezine Articles'
This article talks about the NoFollow tag, its implementation and its impact from the SEO point of view. Also the implementation coupled with the Page Rank characteristics on Google are described here in. >>> More